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ABSTRACT 

Under a program of educational innovation implemented 
by the university, some 20 percent of the faculty were engaged in 
activities designed to improve the guality of instruction • Financial 
support exceeded one-half million dollars. The purpose of this study 
was to determine the program" s overall impact. Baseline data was 
gathered in March 1973, and the survey was replicetjd pne year later. 
Included among the data gathered from students was: (1) expectations 
and actual progress toward the attainment of selected educational 
objectives, (2) activities and interest in the general culture, (3) 
learning styles, and (4) general satisfaction with various 
instructional delivery systems. Each survey includc^d some 4,00Q 
respondents. (Author) 
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Ronald S. Malinski :uk\ Tsc-Ki;} Tchcno ^cZ^^^^^i^^oi'^l^ucy''^' 



SYS'll:^LVrIC STUDI^NT IKPUY INTO liVAMJ-YFIOX OF AN 
L-DUG\riOX;\L INXQVAriOX Pl^OCl^AM 



'Hie ongoing purpose of this project is to uovelop a longitudinal data 
base which would provide for systeiiiatic stud.ent input into the curricuJar 
planning imd evaluation fiuictions of the university. inunediate need for the 
data base was br'ought about by tlie implementation of a prograjn of educational 
and insti^uct ional innovation. Under that prograjii 35 |)roJects were fimded and 
each was sei)arately evaluated. Ihese projects involved inost of the departments 
of the inuversity cUid included as a major thrust eitlier development, iii^:)lenien- 
tation, or evaluation typo activities, (l-or exanrple one project restructured 
three courses into a single competency based course; anotlier project used 
professional actors to present live dnunat Lzations in regular clcisses ; a third 
engaged in an extensive evaluation of the department's principal general 
education coui'se/) In addition, SO instructors received small grants to develop 
innovative instructional practices. Totally, 20 per cent of tlie faculty were 
engaged in activities designed to improve tiie (iuality of instruction. Mnancial 
support exceeded one-half million dollars. The objectives, activities and 
evaluation of these projects are described in more detail elsewhere.'^ 

In addition to any successes enjoyed by tlie Jndi\ddual projects one m.ight 
also exi^ect a t>'pe cf ripple effect. lU'oject faculty taught additional courses 
and the possibility of cariyover existed. Project ideas provided cx^iriples for 
others to emulate. But probably most importcint tlie fonnalized innovation program 
provided an explicit commitment to improving the quality of instruction and 
legitimized sudi endeavors as appropriate for professional activities, llnis 
what appeared necessary was the development of a strategy whicii might get at tlie 
overall impact of the program. 



V^^ PRoniniJi^- 

Baseline data was gathered in March, 1973. Pour fonii^ of a modularized 
questionnaire were developed from measures in the Higher Education Measurement 
^jnd Pvalu ation Kit prepared by the staff of t!ie Higher Pducation Pvaluation 
Progrcim luider the Directorship of C. Robert Pace, [nfonnation of the following 
types was elicited: (1) overall satisfaction with the instructional program; 
(2) general satisfaction ivith various instructional foimats ; (3) expected and 
actual progress toward the attaii^nent of broad educational objectives and benefits 
classified into vocational, general education, critical thinking ai^d human relations 
categories; (4) activities and interest in tlie general culture; (5) student 
characteristics. 232 itcws were divided into su!)grou])s and these were distributed 
among the four ciuest ioimaires so tliat no Jorm would take longer than 15 minutes 
to comjilctc. All undergraduate classes meeting at 9 A^l v»n a IVeclnesday wei^e 
included in tlie sun^ey. Questionnaires were conpleted cinon\TnoiLsl)' during the 
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class period. To increase the likelihood of representativeness the four forms 
\\cre intemingled sequentially when they were packaged for distrihution. Sovi)o 
4000 responses were gatliered for an 80% retuni. The nonrespondents were students 
who were absent or wlio were iji classes wliere the instructor declined tc take part. 
Neither source of bias was considered serious. In particular it appeared that 
the subgroup of students who miss a proportion of their classes were actually 
represented since some of tlie students attending the 9 AM class would likely laiss 
classes at otJier times during t'ne day. 

Tlie original surv^ey with two modifications was replicated in mid-Frebi\iar\', 
1974. (Tlie difference in time of year was due to a clKinge in the sdiool calendar.) 
llie two modifications included elii^dnat ing some subgroups of items iuid distributing 
the remaining cnes cunong three fom^ of tiie questionnaire and administering the 
questionnaire to 10 M classes. Some 4bOO responses were received for im 
approximate return of 75 per cent. 



]y;sui;rs 

CK^erall Satisfaction . If studcjits feel that wiiat they are doing is wortliwliile 
and vi'ev; tJie university favoral^ly, tliey are more likely to be productive and 
to take advantage of the v.WiV opportiniit ics availahdc to tliem. Table 1 summarizes 
tlie results of several atteiiii^t:- to dcteminc globally student feelings regarding 
the iiniversity and tlie quality of education tliey are receiving. Class mccms were 
weighted according to tlie proportion of students in each class based oii actual 
hcadcount. Tlie proportions changed from *75 to ^74 hut in eacli case were in the 
neighborhood of .25. To produce a measure for comi.:)arison puiposes whicli is 
independent of tJie clianging pattern of enrollment, the class means were also 
weighted equally to arrive at a composite value for tlie four ckisses. 

In general, tlie results show a moveiiK^nt to greater satisfaction with tlie 
university altliough tiae movement is s]nall. /vs to be exi^iected, the large sainple 
sizes allow for small observed differences to be statistically significant. As 
yet, we do not have sufficient cxjX'rience wit):) these nieasures to iiiake judgijients 
of practical significance. Tliere are two other obsei*\\ations based on these tables 
uhich you may have noted: first, the results are most favorable for the senior 
cl:iss and secondly, the metliod of weigliting did not make :my material difference 
in the results. 

Informational i^J^'at. Students were asl^ed to clieck which types of instructional 
f ormatT t he\^ 1 1 aTFexpe r i one ed at ISIJ mid their overall degree of satisfaction with 
such experience (Scale: 1 Miglily Dissatisfied to 5 = Highly Satisfied). In 
Table 2 t\vo t\^:)es of data are reported for both 1973 and 1974. First, the per- 
centage of students wlio indicated they liad experienced im instn^ctional foimat at 
least once at the university is shouii. Secondly, of those who had ex^ierienced the 
particular fomat, the mean '^satisfaction" ratings are reported. 



Tcible I. tbiicral Satisfaction of Students With tJic University. 



Item Chiss 113 74 Significance 



1. 


Overall Quality of Instruction 


Er 




.16 




.17 


11 


.s. 




(Scale: 1 = Excellent, 2 = Good, 


SoiVn 




.21 


1 


.21 


n 


. s . 




3 = Fair, 4 = Poor) 


.Jr 


T 


.20 


1 


.16 


n 


.s. 






Sr 








.17=^- 


P< 


.05 




Weiglited f-fean (Ileadcoimt) 






.21 




.18* 


P< 


.05 




Ivcighted lleim (iVeight ^ .25) 






,21 




.18=^ 


P< 


.10 


2 . 


Rclev:uice of Educational Experiences 


]-r 


~) 


, 14 


2. 


, M 


n 


.s . 




(Scale: 1 = Definitely Yes, 2 = Generally 


Sopli 


1 


.20 




, 15 


n 


.s . 




Yes, 5 = Generally No, 4 = Definitely i\'o) 


J r 




,11 




,05=^ 


P< 


.10 






Sr 




, IS 




,08* 


P< 


.01 




Weiglitixl Qleadcoiint) 






, 16 


1 


.10* 


P< 


.001 




Weighted Mean (Ueight = .25) 






,16 


7 


10* 


P< 


.001 


3. 


University's Concern for the Individual 


1-r 




68* 


T 


76 


p<, 


.05 




(Scale: 1 = Definitely Yes, 2 = Generally Yes, 


Sn[^h 


2 


80 


T 


,84 


n, 


. s . 




3 = Generally Ko, 4 = Definitely No) 


Jr 




, 7S 


T 


76 


n, 


, s . 






Sr 




85 


2 , 


,76* 


p<. 


.05 




Weighted ffcan (Headcount) 






78 


2 , 


,78 


n< 


.s. 




Weighted lleim (Weight = .25) 




2 , 


,78 


2 , 


,78 


n< 


,s . 



^Indicates more favoralile response wien difTcrcncc is sigiiif i caiit. 
Approximate saiiplc si::es for eadi ciuestion: 1973 1974 

l"r M=1160 N= 827 

.Sopli N= 955 .N= 970 
.)r N= 980 N=1450 

Sr .\= 720 N=1140 
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Tabic 2. Student rix]:icrioncc ;uk1 Satisfaction with 
Various Instruct ionai Ponnat.s 



Instructional Format 



Per Cent of Students 
Experiencing inst r . Fo una t ^ 
1973 1974 



Mean Satisfaction 

Rating^ 

1973 1974 



Sig. 



Large Lecture Class 

Small Class with Instructor-Led 
Di scuss ions 

Individual Resoarcli as Part of 
- Course Work 

Group Projects as Part of Course 
Work 

Lecture Class witli Scheduled Discussion 
Sections 

A Laboratory Course 

Student-Led Discussion Groups 

Team Teaching 

Group Research as Part of Course 
1-,'ork 

Vidco-Tapcd Lectures 

Courses Involving Coinnuinity Mxperieiict; 

Self-Instructional Packages in Learning 
Laboratories 

Independent Study 

Part or All of Course Work Conducted 
Off Campus 

Residence Hall courses 



99 

97 

89 

82 

81 
75 
74 
70 

6 7 
51 
38 

33 

30 

26 
19 



98 

9 5 

90 

81 

81 
65 
71 
63 

67 
/.S 
36 

34 
31 

32 
18 



2.80 



3.91 



3.08 

3.14 
3.30 
3. 29 
3.40* 

3.01 

2. 32 
3.49 

2.79 
3.39 

3. 28 
2.94 



:.94* p<.001 



3.96 



3.53 3.57 



3.20* 

3. 33* 
3.45* 
3.38* 
3.26 

3.11* 
2.55* 
3.68* 

2.86 
3.49 

3.42* 
3.13* 



n. s. 



n.s, 



P<.01 

P< .001 
P <.001 
P <.05 
P<.01 

P<.05 
P <L .001 
p<.01 

n.s. 
n.s. 

P< .05 
P< .05 



^For 1973, n=1890; for 1.974, n=.l685» Satisfaction ratings wore complet'-.ci only by those 
students v/ho experienced the format at the university. 

^-Scaler i=Highly Dissatisfied, 2-Dissat is f ied , 3=Neutral, 4=Satisfied, 5=}tighly Satis- 
fied, ^indicates higher mean satisfaction rating. 



NOTE: 



Class means were weighted oqua.l.ly to arrive at the overall mean. 



We make the following observations: 



1. Hie most widely experienccci instructional fomats were the large 
lecture type class and tlie small. c1:lss with Instructor led 
discussions. 

2* llie percentage of studejits experiencing a particular fomat did not 
dimige to tmy large extent from '73 to ^74 with the exception tJiat 
there werc 10 per cent fewer students who had experienced a .laboraton^ 
course in IP 74. 

5. TIic .mall class with ins true tor- led discussions received tlie highest 
satisfaction rating on botu years; tlie difference between the two 
years was not statistically significant. 

4. Of the 15 instructional foniiats, 10 received significtintly liighei' 
satisfaction ratings in ' 74, 

Tlie development of mediated instructional materials -and their inplementation 
received lieavy enphasis in the innovation program. Because tlie devclopjiiental 
effort required Is extensive it may ho too early yet to assci^^s tlie impact. However, 
tlie results provide some useful insights for future direction. First of all, 
the signific:int increase in ine:ui satisfaction rating for video-tape lectures i.s 
promising. Ix^ss encouraging is the fact tlnit a similar result did not occur for . 
sol f-iastructional packages in wiiidi mediated materials are a key Ingredient. 
Secondly, the relatively low satisfaction ratings for the \ddeo-tape lecture and 
self-instructional formats indicate tliat the mere production of sudi mediated 
materials will not guarantee acceptaiice liv the students. It seems clear that 
any movement in tlie direction of tediiiolog;v' sliould lie carefully planned and 
coordinated, adequately funded <'ind fully evaluated. Vdiat may appear to be obvious 
is too often lacking in practice. 

ComiriLmity-'based experiences and large lecture classes also received con- 
siderable attention :imong tiie projects funded ruid the evidence is favoral:)le: 
both showed significant increases in sti:dent satisfaction, further, community- 
based exi5eriences were wel 1- rece iir-d by the students as suggested by tlie reUitively 
high mean satisfaction rating. However, with large lecture classes it is a 
different matter, llie relatively low \vci\u rating veiy likely indicates student 
tolerance of sudi classes in general, aiid the question tiien becoiix^s whethci' or 
not quality education can occur luuku' sudi circumstances. Wiile the present 
data does not provide an answer, it does ser\'c to highlight the question. 



Educational Benefits . Tliere were 25 statements^ concenied with educational 
object ives/beneiits associated with colJe^^e. Students were asked to respond to 
these statements in tenns o£ their ''actual" md "preferred" progress toward 
attainment on a scale wliidi ranged from 1 ( Little or None) to (Veiy Mudi) . 
A priority listing for eadi class was deri\c\i by ranking these statements on 
the basis of the mean "preferred progress" ]at„.no. in addition these statciDents 
were also ranked on the basis of the moiui "actiuil progress" rating. The 
difference between tJie two means for each statement c:m be view^ed as a measure 
of discrepanc)' for that particular ol\jectivc/beneflt . Hdited versions of these 
statements appear in Table 4. 

llie magnitude of the rank-order correlations (Table 5) indicate that no 
large dianges in ranks occured in the 1974 survey for the Preferred and Actual 
progress dimensions. The somewhat lov.or correlation between the junior class 
Discrepancy rankings for the two >'oars was due primarily to the relatively 
smaller discrepancies for certain of the himum relations benefits in 1974. Tlie 
correlations between tlie Actual and P rcfe rr ed ratings sliow that in general 
students tend to perceive themselves as mafaiTg relatively more progress in tliose 
areas whidi are viewed as more impoi-t:uit to them. 



Table 5. Il:mk-Order Correlations of I^fcan Ratings for the 
25 Statements of Selected Huucational Benefits 
by ckL^s 



I'r. Sopli. Jr. Sr. 



Preferred progress ratings: 1973 vs. 1974 
Actual pTogress ratings: 1975 vs. 19 74 
Discrepanc)^ ratings: 1973 vs. 19^4 

1973: Preferred vs. Actual ratrnv;-^ 
1974: Preferred vs. Actual ratings 



.91 


.91 


.97 


.90 


.96 


.98 


.91 


.95 


.92 


.94 


.76 


.88 


.61 


.51 


.66 


.67 


.72 


.62 


.71 


.71 



N'arii: For n = 25, p<.01 for r ^.51 (two-tailed test) 

Tlie mean of the rank-order correlations among tlie four .1 hisses for the 
Preferred, Actua l and Discrep rmcy nuiklngs were .89, .95 and .38. Because of this 
tlie means^ for each class were weighted equally to arrive at an overall mean for 
each of the three dimensions for both 1975 :md 1974. The ranks of these means 
are reported in Tal'le 4. 



Tabic 4. l^anks of Educational l^encfit Statements for Preferred 
Progress, Actual Progress and Discrepanc>v 



Preferred Progress Actual Progress Discrepancy 
Statements by Category 1973 1974 1973 1974 1973 1974 

Rank Rank Rank Rank Rank Rank 



Human Relations 



Development of an identity 


1 


1 


8 


6 


3 


3 


Social development 




2 


2 


2 


17 


17 


Personal tlevelopment 


4 


3 


4 


3 


16 


20 


Tolerance of others 


3 


4 


1 


1 


23 


23 


Development of a personal philosophy 


17 


16 


15 


14 


13 


14 


Vocational 














Background for further education 


7 


6 


10 


11 


9 


8 


Vocational training 


9 


9 


25 


25 


1 


1 


Discovery of vocational intersts 


15 




21 


21 


4 


c. 
0 


Critical Thinking — to develop 














Open-mindedness 


c 

J 


c 
J 


Q 
J 




ZU 


1 Q 


Intellectual curiosity 


0 


7 


Q 


Q 

y 


1 n 

iU 


J.l 


Ability to select appropriate information 


o 


Q 
O 


1 J 


13 


o 
o 


y 


Intellectual honesty 


13 


1 1 


12 


12 


12 


12 


Desire for order 


16 


15 


14 


15 


14 


13 


Ability to recognize assumptions 


on 


1 Q 

lo 


ZU 


ID 






Ability to define problems 


14 


14 


18 


18 


7 


5 


General Education 














Current issues h proble-.ms in society 


10 


13 


19 


20 


2 


4 


Develop interests in ne^' fields 


12 


12 


7 


7 


15 


18 


Terminology h facts In various fields 


11 


10 


6 


8 


19 


15 


Awareness of different cultures 


18 


20 


5 


5 


24 


25 


Effective communication 


19 


17 


22 


22 


5 


2 


Appreciation of moral & ethical standards 


21 


21 


11 


10 


25 


24 


Quantitative thinking 


22 


22 


24 


23 


6 


7 


Broadened literary appreciation 


23 


23 


17 


17 


21 


21 


Aesthetic sensitivity 


24 


24 


16 


19 


22 


22 


Understanding the nature of science 


25 


25 


23 


24 


18 


16 



NOTE: Means ranked according to numerical value from High = 1 to Low = 25. 



Tliere arc several observations which can be made from the table. 

1, The Human Relations typo benefits such as development of an identicy, 
social development, personal development and tolerance of others ranked 
highest in priority. Certain benefits of a general college education 
well accepted by faculty, such as, development of quantitative thinking 
skills, broadened literary appreciation, aesthetic sensitivity, and 
understanding the nature of science, ranked lowest in priority. 
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2. Perceived actual progress was greatest for tlie following: tolerance 
of others, social development, personal development, open-mLndedness 
and awareness of different cultures. Perceived actual progress was 
least for tlie following: vocational ti^aining, iniderstanding tlie nature 
of science, quantitative thinking, effective coniniuiucation and 
discovery of vocational interests. 

3. Among the largest discrepancies were included the following: vocational 
training, development of an identity, curi^ent issues and problems 

in society, and effective communication. (To place the magnitude of 
these discrepancies in perspective, the largest value was 1.33 for 
"vocational training" while the smallest was .59 for "awareness of 
different cultures". Across all objectives tlie marked tendency is 
for students to rate "preferred progress" liigher than "actual progress".) 

It is possible for gains to occur :ind have the r:iiiks remain relatively 
stable, llius, it miglit be well to consider the differences in mecui ratings between 
the two surveys. Based on equal weighting of class me:ms to arrive at an overall 
mean> significant increases (P<.05) were obsei'i^ed in Actual progress on the 
following: (a) Personal development- increfise from 5.32 to 3.59, (b) Development 
of an identity- increase from 3.15 to 3.24^ (c) Quantitative thinking-increase 
from 2.72 to 2.81. There were no significant decreases nor were :my of the 
differences between the mean discrepancies signi fic:mt. 

Ivlien the differences in me^m discrep^mcies were tested by class, four were 
significantly lowered (p<.05). However, when you consider 100 tests were ixin, 
little, if any, importance can be attached to these. Ry class, tliere were 27 
differences in "Actual progress" means whidi were significant (p<.05). Ten of 
these indicated greater progress in tlie ^73 sui'vey imd they occuri^ed in total for 
the freshman class. Tlie remaining seventeen of the signi ficmit differences indicated 
greater progress in the '74 survey and were dissociated primarily with the junior 
and senior classes. The majority w^ere in the general education and human relations 
area. By tlie nature of the project funded it would be difficult to attribute the 
gains in the himian relations categor\' to the fomialized prograni especially since 
there is a currently strong movement for establishing local diapters of national 
social frateniities and sororities. 

Our experience in dealing with faculty members regarding evaluation of their 
particular projects demonstrated to us tlie difficulty in discussing instructional 
objectives • Inevitably such attempts would lapse into discussions of process, that 
is, instructional fomat. It is our conjecture at tJris point that tlie gains 
exliibited in student satisfaction witli ins tiaictional format reflect tliis emj-^liasis 
and the lack of similar t\i)e gains regarding "Actual progi^ess" or decreasing 
the discrepancy values reflects tlie lack of emphasis on instiTictional objectives. 

Student Qiaracteristi.es. Ihe natiu'e and quality of learning a student engages in 
is ref Jccted t"o" large extent by behavior during lectures and periods of study, llie 
results in Table 5 indicate that students took notes during lectures primarily for 
tests as opposed to taking notes ior their out. interests. Ilwever, from ^75 to ^74 
there were significant differences in the direction of higher quality leaniing 
behaviors. Again, to put the mean values in perspective, in 1974, 7^0 of the 
students indicated tJiat "very fretiuently" they took notes primarily for tests opposed 
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to 26^^ who indicated tliat "ver>" frequently" they took notes primarily for their 
am interests. Regarding periods of study there were no differences in behavior 
indicated between * 73 and '74 with tlie exception that students appeared to spend 
nore time tliinking about applications of w'nat they were studying. 



Table 5. Student Behavior During Lecture Classes 
and Study Periods 



1973 Mean 
(N=950) 



1974 Mean 
(N=2630) 



Significance 



During Lectures: 

Take notes primarily for tests 2.68 

Take notes primarily for personal interests 1.96 

Relate what instructor says to other things 2.21 



2.58* 
2.05'V 
2.26^^ 



P < ,001 
p < .001 
P<.05 



During Study Periods: 

Read assignments without understanding them 
Memorize facts 

Relate concepts to personal experience 
Think about applications of the material 



1.58 
2.11 
2.16 
2.12 



1.56 
2.09 
2.20 
2.22* 



^Scale: 1 = Seldom or Never; 2 = Often; 3 = Very Frequently 
^Indicates more favorable value 



n. s . 
n. s . 
n . s . 
p < .001 



Tlie extent to whicli individuals engage in certain kinds of activities is 
a reflection of their interests and attitudes. Eight brief activity scales 
related to broad general education objectives were selected from the KIT^for 
inclusion in the survey, lliese scales sajnpled behaviors which range from common- 
place activities to those wlucli require niore effort and thus imply a more intensive 
level of involvement. On eac'n scale students were asked to check those actiAdties 
tliey had engaged in during the past >'car. The score for a given scale was the 
number of items diecked. The results are presented in Table 6. One paitiicular 
application of tiiese scales whicli is appealing is tlieir ase iis a pre- and post- 
measure in the evaluation of individual projects or courses. Tlie availability of 
caii]}ous-wide noniis for appropriate identifiable subgroirps could sen^e as a partial 
solution to the problem of setting standards in the fomiation of goals. Sucli 
norms iiught be viewed as minimal standards. 



ERIC 



- 9 - 



Tciblc 6. Activities and Interest in tlie General Culture 



Numl^er _ 1975 1974 

Scale or Items N Mean N Mean Sig. 



Art 


7 


769 


2. 


98 


712 


2. 


, 96 ■ 


n. 


s . 


Music 


9 


931 


6. 


,08 


850 


6. 


,02 


n. 


s. 


Literature 


S 


852 


5. 


.84 


773 


3. 


,76 


n. 


s. 


Conmuiity /Vi fairs 


7 


842 


2. 


32 


721 


7 


,36 


n. 


s. 


Drajua 


7 


902 


3, 


,90 


819 


4, 


02* 


P<. 


10 


International ^ Intercultural AfTairs 


n 

D 


808 


3, 


,17* 


721 


2. 


,95 


P <• 


,01 


Science 


10 


788 


5. 


,18* 


677 




,98 


P <■ 


,05 


National and State Politics 


10 


814 


3. 


57 


774 


/I 


,01* 


P<- 


001 



* Indicates more favorable value 



Tor students to be deeply involved in academic work and to become independent 
learners is considered by many facult)' to be tbc essence of a college education. 
In a speculative vein t\<o related scales from the KIT v;ere included. Ihe first, 
titled Style of Leanii ng- Academic , is described as measuring ''the style of one's 
effort in relation to the acquisition of loiov/ledge md understanding from courses 
and readings, such as participation in class discussions, talking with professors, 
devoting concentrated periods of time to academic work and reading related but 
unassigned work." Based on equally weighted class mecms the level of participation 
in academic life had increased signi fic^mtly (p<.001) among the undergraduates. 
Tne second scale, titled Intellect ual Orientation , purports to measure the 
disposition of cm individual towafcPO^e creation, development and application of 
new ideas and the preference for inde[*jendcnt tliought. In this case neither the 
difference between tl^e weighted means nor tliose loetwcen the class means for the 
two s u r\'e)'s we re s i gni f i c^m t . 
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Generally, the concer^i of institutional reseai^di has dealt with the economic 
and acMiustrative aspects of the university. U'e are advocating the systematic 
study of the educational effects of intervention activities as cin equally imi^ortant 
research fiQiction. Additioiic-lly it is our contention tliat the procedure we have 
employed is vial)le during this period of tight budgets. 

Ivhile the results to date arc encouraging, we are veiy much aware of the pit- 
falls in attemi:)ting to attribute the positive gaiiis directly to the formalized 
I^rogram of innovation. M^ile studies of this ty^^e are not rigorous in the ex]ieri~ 
mental sense md quite sasceptible ro criticism, they are necessaiy and valuable. 
To be sure, there are difficult tcclmical prol;)lems. For excimple, how valid is 
the notion of perceived actual progress as a proxy measiu'e for tlie more direct 
assessment of the outcomes of higher education? Xo less inportant is the problem 
of faculty acceptance. In tlie original evaluations of the projects it was virtually 
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impossible to have faculty talk in ten'i> of student oiitcon^es. TlxC seemingly 
unfamiliarity with the notion along with a distiiist of hoiKivioral i:>c>asuren)ent 
techniques were major contributing factors. To oftx?ctively deal with sudi prd:^leiiis 
and to bring about the necessaiy methodological rofinenents grcater experience witli 
longitudinal data of the t>X'e presented h.eiv, along uiti> the sd^se-viuent disseniin- 
ation and discussion of its iiiij)iicatiorLS , is needed. 
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^See "Innovation at Illinois State liiivei-sity'^ HRiC niunber: ELX)S2694. 
(A limited nioiiber of copies are available tvo::\ tlvc first author-) 

•^llie statements were cisseirMed fror a variety of sources; however, the 
major source wiis the {iigher H ducat ion 'k? asu?v?::ent 5^ H valuation Kit developed 
inider tiie directorship of C. Robert ?ace at UCLA, Center for the Study of 
Evalu^ition. 

^KIT refers to the tligiicr I'dueati on -^^^^^'^iiei^it 5 Hvaliiation Kit. 
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